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Abstract 

Amethodfor 1 ocal i zati on and posi ti oni ng i n an i ndoor environment is presented. Localizat’, 
is the act of recognizing the environment, and posi ti oni ng i s the act of computing the e 
coordi nates of a robot i n the envi ronment. The method i s based on representi ng the so 
as a set of 2D vi ews and pr edi ct i ng t he appear ance of novel views by linear combi nati or 
the model views. The met hod accur at el y appr oxi mat es the appearance of scenes under weak 
perspecti ve pr oject i on. Anal ysi s of t hi s pr oj ect i on as veil as exper i ment al r es ul t s deni 
t hat i n many cases this appr oxi mat i on i s s uffici ent to accur at el y des cr i be t he scene. 1 
orthographi c approxi mati on i s i nval i d, ei ther a 1 arger number of model s can be acqui red i 
i terati ve sol uti on to account for the perspecti ve di storti ons can be enpl oyed. 

The pr es ent ed met hod has s ever al advant ages over exi s t i ng met hods . It uses r el at i vel y 
r epr es ent at i ons , t he r epr es ent at i ons ar e 2D r at her t han 3D, and 1 ocal i zat i on can be done 
a si ngl e 2D vi e w oni y. The s ame pr i nci pal met hod i s appl i ed bot h f or t he 1 ocal i zat i on as 
as the posi ti oni ng probl ent, and a si npl e al gori thmfor repositioning , the task of returni 
a pr evi oust y vi si t ed posi t i on defined by a s i ngl e vi e w, is der i ved f r omt hi s met hod. 
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1 Introduction 


Basi c t as ks in aut ononous robot navi gat i on are 1 ocal i zat i on and posi t i oni ng. Local i zat i or 
act of r ecogni zi ng t he envi r onment, t hat is, as si gni ng consi s t ent 1 abel s t o di ffer ent 1 ocal 
posi t i oni ng is t he act of conput i ng t he coor di nat es of t he robot i n t he envi r onment. Posi t i 
is a t as k conpl ement ar y t o 1 ocal i zat i on, i n t he s ens e t hat position(e.g., “1.5meters no 
of t abl e T”) is of t en s peci hed i n a pi ace - s peci he coor di nat e system(“in room 911”). I 
paper we suggest a method of both 1 ocal i zati on and posi ti oni ng usi ng vi si on al one. Aval 
of the posi ti oni ng pr obi erg referred to as repositioning , involving the return to a pre' 
vi si t ed pi ace i s al so di s cus sed. 

Pr evi ous s t udi es have exam ned t he pr obi era of 1 ocal i zat i on and posi t i oni ng under a va: 
of conditions, defined by the ki nd of sensor(s) employed, the nature of the environment, 
t he r epr es ent at i ons us ed. W can di s t i ngui sh between active and pas si ve s ensi ng, i ndoo: 
outdoor navi gat i on t as ks , and metric and t opol ogi cal representations. The metric appi 
at t e npt s toutilize adetailedgeometric description of the e nvi r onme nt, whi le the topo 
approach uses a nor e qual i t at i ve des cr i pt i on i ncl udi ng a gr aph wi t h nodes r epr es ent i ng p 
and arcs representi ng sequences of acti ons that woul d resul t i n novi ng the robot fromone ] 
t o anot her. 

In the paper we consi der a robot that uses a passi ve sensor , vi si on, i n an i ndoor envi ron 
The envi r onment cannot be changed by t he robot t o i npr ove i t s performance; nei t her beacon 
nor floor or wall markings are employed. The paper addresses both the 1 ocal i zati on and i 
posi t i oni ng pr obi era . Sol ut i ons to t hes e pr obi era are pres ent ed bas ed on obj ect r ecogi 
techniques. The method, based on the linear combinations scheme of [17], represents see 
by sets of their 2D images. Tocalization is achieved by comparing the observed image 
linear combinations of model views, and t he pos i t i on of the robot is conput ed by anal yi 
t he coeffci ent s of t he 1 i near combi nat i on t hat al i gns t he model t o t he i mage. M so, a si i 
“qualitative” solution to the repositioning problemusing the linear combinations sell 
presented. 

The rest of the paper i s organi zed as f ol 1 ows . The next secti on descri bes the 1 ocal i zat 
posi t i oni ng pr obi em and s ur veys pr e vi ous s ol ut i ons . The net hod of 1 ocal i zat i on and posit 
usi ng 1 i near combi nat i ons of model views is describedinSection3. The net hod ass unes w 
perspective projection. Aniterative sc he me to account for perspective distorti ons i s p 
inSection4. An anal ysi s of t he er r or r es ul t i ng f r omt he pr oj ect i on as s umpt i on i s pr es e 
Secti on 5. Cbns t r ai nt s i npos ed on t he not i on of t he r obot as a r es ul t of s peci al pr oper t 
i ndoor envi r onment s can be us ed t o reduce t he conpl exi t y of t he net hod pres ent ed her e. Th 
t opi c i s cover ed on Sect i on 6. Exper i rnent al r es ul t s f ol 1 ow. 
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2 The Problem 


Local i z at i on and posi t i oni ng f r omvi s ual i nput are defined in t he f ol 1 owi ng way: Q ven a 
ni 1 i ar environment, identify the observed environment, and then find your position in i 
envi r onment. Local i zat i on r es embl es t he t as k of obj ect r ecogni t i on, wi t h obj ect s r epl i 
s cenes . Once 1 ocal i zat i on i s acconpl i s hed, posi t i oni ng can be per f or med. 

One probl ema systemf or 1 ocal i zati on and posi ti oni ng shoul d address i s the vari abi 1 
i mages due to vi ewpoi nt changes. The i nexactness of practi cal systems makes i t di ffcul t f 
r obot to return to a specified position on s ubs equent vi si t s . The vi s ual dat a avai 1 abl e 
robot between vi si ts vari es i n accordance wi th the vi ewi ng posi ti on of the robot. A1 ocal 
syst emshoul d be abl e t o recogni ze scenes f romdi ffer ent posi t i ons and ori ent ati ons . 

Anot her probl emi s t hat of changes inthescene. At s ubs equent vi si t s t he s ame pi ace r 
1 ook di ffer ent due t o changes i n t he ar r angement of t he obj ect s , t he i nt r oduct i on of new obj 
and t he removal of ot her s . I n gener al , s one obj ects tend to be more s t at i c t han ot her s . \ 
chai r s and books ar e of t en moved, t abl es, closets, and pi ct ur es t end t o change t hei r po 
much less, and wal Is are al mos t guar ant eed tobestatic. St at i c cues nat ur al 1 y ar e mor ere 
t han mobi 1 e ones . Confini ng t he s ys t emt o s t at i c cues , however , may i n s one cases result 
f ai 1 ur e to r ecogni ze t he s cene due t o i ns uffci ent cues . The s ys t ems houl dtherefore at ten 
r el y on st ati c cues , but shoul d not i gnore t he dynani c cues . 

Sol ut i ons t o t he pr obi e mof 1 oc al i z at i on f r omvi s ual dat are qui re a 1 ar ge me mor y and h< 
conput ati on. Exi sting systems often try to reduce t hi s cost by usi ng s par s e repr es ent; 
and by expl oi t i ng cont ext ual information. Spar se represent ati ons ar e i nt roduced i n [ 1 
Mrt aric [10] represents scenes as s equences of 1 andmar ks (such as wal is, doors, etc.) ex 
by t r aci ng t he boundar i es of t he s cene usi ng a s onar and a conpas s . Mt r i c i nf or mat i on 
and bet ween t he 1 andmar ks i s not s t or ed. Sar achi k [14] r ecogni zes a room by its dimensi 
whi ch ar e me as ur ed by i dent i f yi ng and 1 ocat i ng t he t op cor ner s of t he r oomus i ng stereo ( 
(obt ai ned f r omf our earner as ). I n bot h cas es t he r epr es ent at i on i s ver y s par s e, and t he s c 
t her ef or e of t en ambi guous . 

Ri cher r epr es ent at i ons ar e us ed i n [ 3, 5] where hi gher success rates are rep or ted. Br; 
[3] r epr es ent s t he scene by an occupancy t abl e, a 2D bi t ar r ay whi ch cont ai ns a 1 at e\ 
1 ocat i on occupi ed by s one obj ect. The t abl e i s cons t r uct ed by t aki ng stereo pi ctures co’ 
360P f r omt he ni ddl e of t he room and pr oj ect i ng t he obt ai ned 3D dat a ont o t he floor . The 
met hod s uffer s f r oml os s of i nf or mat i on due to t he pr oj ect i on ont o t he floor . 

Engel son et al . [5] represent the scene by a set of invariant “signatures”. Asignat 
us ual 1 y compos ed of 1 ow- r es ol ut i on gr ay-1 evel or r ange dat a obt ai ned by bl ur r i ng an i mag 
set of si gnat ur es t aken f r omdi ffer ent vi ewpoi nt s ar e s t or ed. A s cene i s r ecogni zed i f t h< 
encount er s a s i gnat ur e si ni 1 ar to one of the storedsi gnat ur es . 

Sys t ems t hat use t he f ul 1 i nf or mat i on pr ovi ded by t he i mage (e. g. , [6, 12]) us ual 1 
on cont ext ual i nf or mat i on t o avoi d s canni ng al 1 t he model s i n t he me mor y and t o reduce t 
conput at i onal cost of conpar i ng a model t o t he i mage. The s ys t emf ol 1 ows a pr edet er ni ne 
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pat h, sot hat t he i dent i t y of each vi si t ed 1 ocat i on i s known i n advance, and 1 ocal i zat i on l 
a veri hcati on probl em Path conti nui ty i n nany cases i s essenti al , and the so- cal 1 ed “drt 
pr obi emi s not addr es s ed. The enphasi s inthese s ys t ens is onpositi oni ng, whi ch i s use 
keep t he r obot on t he pat h. 11 i s t ypi cal for these systertE (e. g. , [1, 6, 12]) to use a 
model of the environment. 

Onoguchi et al. [ 12] , among ot her s , r epr es ent t he envi r onment by a s et of 1 andmar ks s el e 
f r ompai rs of stereoi mages by a human ope r at or . Thes e 1 andmar ks ar e t r ansf or med by an i mag 
processi ng program whi ch i s desi gned so as to i denti f y the sped he 1 andmark usi ng spe 
extracti on i nstructi ons (suchas what features tolookfor and at what locations). Local 
is achieved by applying the extraction procedure specihed for the next landmark. Che 
1 andmark is i dent i hed, t he posi t i on of t he robot r el at i ve to t hat 1 andmark is det er ni n 
compar i ng t he di mens i ons of t he obs er ved 1 andmark withthose of the stored model. 

The met hod presented i n thi s paper r epr es ent s t he envi r onment usi ng a s et of edge map 
Local i zati on and posi ti oni ng are achi eved by compari ng i mages of the envi ronment to 1 i 
combi nati ons of the model vi ews. The method uses ri ch vi sual i nf ormati on to represent 
scene. The systemis flexible. In many cases it is capable of recognizing its locatio 
one i mage oni y ( 3 6 Ot over age is not r equi r ed). Wen one i mage i s not s uffei ent, addi t i ona 
i mages can be acqui r ed t o s ol ve t he 1 ocal i z at i on pr obi em Cbnt ext can be usedto deterr 
t he or der of compari s on of t he model s t o t he obs er ved i mage and toincrease the confidence 
a gi ven mat ch, but cont ext i s not es s ent i al : t he s ys t emcan al s o, by per f or ni ng mor e ext e: 
computations, solve the “drop-off” problem 

3 The Method 

The probl ems of 1 ocal i zati on and obj ect recogni ti on are si ni 1 ar in many ways . Both probl 
require the matching of visual images to stored models, either of the environment or o: 
obs er ved obj ect s . Bot h probl ertE f ace si ni 1 ar di ffcul t i es , s uch as var yi ng i 11 uni nat i on con 
and changes i n appearance due to vi ewpoi nt changes . Si ni 1 ar methodol ogi es therefore cai 
us e d f or s ol vi ng bot h pr obi e he . 

Aparticular appl i cat i on of an obj ect recogni t i on s cheme, the Linear Combinations ( 
sc he me [17], tot he probl ertE of 1 ocal i zat i on and pos i t i oni ng i s di s cus s ed bel ow. The envi r 
is representedinthis scheme by a s mal 1 set of vi ews obt ai ned f r omdi ffer ent vi ewpoi nt s ai 
t he cor r es pondence bet ween t he vi ews . A novel vi ew i s recogni zed by compari ng i t t o 1 i i 
combinations of the stored views. Positioning is achieved by recovering the position 
carter a relative to its positioninthe model views fr omt he coeffci ent s of t he al i gni ng 
combi nat ion. I n t he r es t of t hi s s ect i on we r evi ewt he 1 i near combi nat i ons approach and des 
i t s appl i cat i on t o bot h 1 ocal i zat i on and posi t i oni ng. The sect i on concl udes wi t h a s ol i 
t he pr obi emof re posi t i oni ng, t hat is, t he probl emof r et ur ni ng to a pr e vi ousl y vi si t ed 
by “1 ocki ng” i nto an i mage acqui redin that posi ti on. 
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3.1 Localization 


The probl emof 1 ocal i zati on i s defined as f ol 1 ows : gi ven P, a 2Di nage of a pi ace, and Ai, a t 
storednodels, find a model* JifAl sucht hat P nat ches M Local i z at i on i s t he r ecogni t i on 
of a pi ace. It can t her ef or e pot ent i al 1 y benefit f r omusi ng an obj ect r ecogni t i on met hod 
A comnon approach t o handl i ng t he problemof recogni ti on f romdi fferent viewpoints is b 
conpari ng the stored no del s to the observed envi ronment after the vi ewpoi nt is recovered 
conpens at ed f or . This approach, cal 1 ed al ignment , is used i n a number of studies of oh 
recogni ti on [ 2, 7, 8, 9, 15, 16] . W appl y the al i gnment approach to the probl emof 1 ocal i 5 
The s ys t emdes cr i bed bel owus es t he “Li near Cbirbi nat i ons ” (LC) s chene, whi chms suggestei 
by 111 nan and Bas r i [17]. 

W begi n vi th a bri ef revi ewof the LCschene. LCi s defined as f ol 1 ows . Q ven an i nage, wr 
cons truct two vi ewvectors f r omt he f eat ur e poi nt s i n t he i nage, one cont ai ns t he x- coor di 
of t he poi nt s , and t he ot her cont ai ns t he y- coor di nat es of t he poi nt s . An obj ect (in oui 
t he envi r onnent) is nodeledbyaset of suchviews, where t he poi nt s i n t hes e vi ews are or di 
in correspondence. The appearance of a novel vi ew of the object is predicted by appl } 
1 i near combi nat i ons to t he s t or ed vi e ws . The pr edi ct ed appear ance is t hen conpar ed wi t h 
actual inage, and the object is recognized if the two natch. The advant age of this netl 
i s t wof ol d. Fi r s t, vi ewer - cent ered r epres ent at i ons ar e us ed r at her t han obj ect - center 
nanel y, model s are composed of 2Dviews of the observed scene; second, novel appearances ; 
predictedinasi mpl e and accur at e way (under weak per s pect i ve pr oj ect i on). 

Formal 1 y, gi ven P, a 2Di nage of a scene, and Ai, a set of stored model s , the obj ecti ve : 
find a model M £ Ms uch t hat P =£j = i cijMj f or s one cons t ant § 6 1Z. It has been s hown 
that this scheme accurately predicts the appearance of rigid objects under weak perspe 
pr oj ect i on (or t hogr aphi c pr oj ect i on and s cal e). The 1 i ni t at i ons of t hi s pr oj ect i on m 
discussedlater inthis paper. 

M>re concretely, 1 f t=^ Xi, yi, %), 1 < i < n, be a set of n object points. Under weak 
perspective projection, the po^i=t (o|i, ^r) of these poi nt s i n t he i nage are gi ven by 

x'i = sr n x t + s r 12 y t +s r 13 z t +t x 

Vi = sr 21 x t +sr 22 y t +sr 23 z t +t y (1) 

where jjj are t he component s of a 3 X 3 rot at i on nat r i x, and s is a scale factor. Rewriting 
i n ve c t or e quat i on f or mwe obt ai n 

x' = sr n x+sr i 2 y +sr 13 z +t X 1 

y' = sr 21 x+sr 22 y+sr 23 z+t y l (2) 

where x, y, z,'^.f E R are the vectors c$ ^ and y[ coordinates respectively, and 

1 = (1, 1, .. . , 1). Cbnsequent 1 y, 

x', Zf £ span{x, y, z, 1} (3) 
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or , i n ot her wor ds^axd y' bel ong t o a f our - di mens i onal 1 i near s ubs pSce(cNbt7i’.ce t hat 
z', t he vect or of dept h coor di nat es of t he pr oj ect ed poi nt s , al s o bel ongs t o t hi s s ubs pac 
f act is us ed i n Sect i on 4 bel ow. ) A f our - di mensi onal s pace i s s panned by any f our 1 i m 
independent vectors of the space. Two vi ews of the scene supply four such vectors [13, 
Etenot e by x[, y and x 2 , y> t he 1 ocat i on vect or s of the n poi nt s i n t he t wo i imges ; t hen t her 
exist coeffci entp, ag, <g, q and b, |, k such that 

x' = a ixi +a 2 yi +a 3 x 2 +a 4 1 

y' = b rxi +b 2 yi +b 3 x 2 +b 4 1 (4) 

( Nat e t hat t he vect of yl ready depends on t he ot her f our vect or s . ) Si nee R i s a r ot at i o 
nat r i x, thecoeffcients satisfythefoll owi ng t wo quadr at i c cons t r ai nt s : 

aj+al+al- bj- §- l=2(b 1 b 3 - qa 3 ) r n + 2 ( b 2 b 3 - qa 3 )r 12 

a\b\ +a 2 b 2 +a 3 b 3 +( a i & 3 +a 3 &i) rn +( a 2 b 3 +a 3 b 2 ) r 12 =0 (5) 

To der i ve t hes e cons t r ai nt s t he t r ans f or nat ionbetweenthetwomodel views s houl dbe recove 
Thi s can be done under weak per spect i ve usi ng a t hi r d i nage. M t er nat i vel y, t he cons t r f 
can be i gnor ed, i n whi ch cas e t he s ys t emwoul d conf us e r i gi d t r ansf or nat i ons wi t h affhe or 
Thi s us ual 1 y does not pr event s ucces s f ul 1 ocal i zat i on s i nee gener al 1 y s cenes ar e f ai r 1 3 
fromone another. 

A LC scheme for the probl emof 1 ocal i zati on i s as f ol 1 ows : The envi ronment i s model 
by a set of images with correspondence between the images. For example, a spot can b 
model ed by t wo of its cor r es pondi ng vi ews . The cor r es pondi ng quadr at i c cons t r ai nt s nay f 
be stored. Localizationis achievedby recove r i ng t he 1 i near combi nat i on t hat al i gns t In 
to the observed image. The coefficients are determined using four model points and the 
correspondi ng i nage poi nt s by sol vi ng a 1 i near set of equati ons . Three poi nt s are sufffci < 
determine the coefficients if the quadratic constraints are also considered. Additional 
nay be used to reduce the effect of noi se. 

The LC scheme uses vi ewer-centered model s , that is, representations that are conpot 
of images. It has a number of advantages over methods that build full three-di mensi c 
model s to represent the scene. First, byusingviewer-centeredmodels that cover rel ati vel 
t r ans f or nat i ons we avoi d t he need t o handl e occl usi ons int he scene. If fromsome vi ewpo 
the scene appears different because of occlusions we utilize anewmodel for these vi ewpc 
Second, viewer-centeredmodels are easier to bui 1 d and t o nai nt ai n t han obj ect - centered 
The model s cont ai n onl y i mages and cor r es pondences . Bylinitingthetr ansf or nat i on bet we 
t he model i mages one can find t he cor r es pondence usi ng mot i on met hods . If 1 ar ge por t i ons 
t he envi r onment ar e changed betweenvisits anewmodel can be cons t r uct ed by si npl y r epl aci 
old images withnewones. 

One pr obi emwi t h us i ng t he LC sc he me for localizationis due to the weakperspective a 
proxi nati on. In contrast wi th the probl emof obj ect recogni ti on, where we can general 1 y as 
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t hat objects are s iml 1 r el at i ve t o t hei r di s t ance f r omt he earner a, in 1 ocal i zat i on t he 
ment surrounds the robot and perspective distortions cannot be neglected. The 1 i ni t at 
of weak per s pect i ve model i ng are di s cus s ed bot h mat he mat i cal 1 y and enpi r i cal 1 y i n t he ] 
two secti ons . It i s shown that i n many practi cal cases weak per spec ti ve i s s uffei ent toe 
accurate localization. The main reason is that the problemof 1 ocal i zati on does not r 
accur at e me as ur ement s i n t he ent i re i mage; it onl y r equi res i dent i f yi ng a s uffei ent numb 
spots to guarantee accurate nani ng. If these spots are relatively close to the center 
i mage, or if t he dept h di ffer ences t hey cr eat e ar e r el at i vel y s mal 1 (as inthe case of lo< 
a val 1 when t he 1 i ne of si ght is near 1 y per pendi cul ar t o t he val 1 ), t he per s pect i ve di s' 
are rel ati vel y smal 1 , and the system can i denti f y the scene vi th hi gh accuracy. M so, 
related by a tr ansi at i on par al 1 el t o t he i mage pi ane f or ma 1 i near s pace even when per s pe 
di st orti ons are 1 arge. Thi s case and ot her si npl i heati ons ar e di s cus s ed i n Sect i on 6. 

By usi ng weak per s pect i ve ve avoi d s t abi 1 i t y pr obi ent t hat f r equent lyoccur inperspei 
conput at i ons . W can t heref ore conput e t he al i gnment coeffci ent s byl ooki ng at a rel ati v 
narrowhel d of vi ew. The enti re scheme can be vi eved as an accumrl ati ve process . Rather th 
acqui r i ng i mages of t he ent i re s cene and conpar i ng t hemal 1 t o a f ul 1 s cene model (as ir 
ve r ecogni ze t he s cene i mage by i mage, spot by spot, unt i 1 ve accumrl at e suffei ent convi nci 
i nf or mat i on t hat i ndi cates t he i de nt i t y of t he pi ac e. 

\Hen perspective distortions are relatively large and veak perspective is insuffeie 
model t he envi r onment, t wo approaches can be us ed. One possibilityis to cons t r uct ala 
number of mode is so as tokeepthepossible changes bet veen t he f ani 1 i ar and t he novel vi ( 
s mal 1 . Al t er nat i vel y, an i t er at i ve conput at ion can be appl i ed t o conpens ate for these c 
t i ons . Such aniterative methodis describedin Secti on 4. 

3. 2 Posi t i oni ng 

Posi t i oni ng i s t he pr obi emof recove r i ng t he exact posi t i on of t he robot. Thi s posi t i on 
s peci hed i n a fixed coor di nat e s ys t emas sociatedviththe envi r onment (i . e. , r oomcoor di n; 
or it can be associatedviths ome model , in whi ch cas e 1 ocat i on i s expr es sedvithrespect 1 
posi t i on f r omwhi chthe model vi e vs vere acqui red. In thi s secti on ve di scuss an appl i c; 
of t he TC s cheme t o t he posi t i oni ng pr obi em 

The i dea i s t he f ol 1 ovi ng. W as s ume a model compos ed of tvu i mjrg0$4 t hei r 
r el at i ve posi t i on is gi ven. Q ven a novel nwgehrfi t al i gn t he model vi t h t he i mage 
(i . e. , 1 ocal i zat i on). By consi deri ng t he coeffci ent s of t he 1 i near combi nat i on t he r obot ’ 
rel ati ve to the model i mages i s recovered. To recover the absol ute posi ti on of the robot 
r oomt he abs ol ut e posi t i ons of t he model vi evs s houl d al s o be pr ovi ded. 

As s uni ng i 2 i s obt ai ne d f r orpPy a r ot at i on R, t r ans 1 at i on J, and s c al i ng s , t he 
coor di nat es of a poi nt i,n(Jj jj ), can be wri 11 en as 1 i near combi nat i ons of t he cor r es pondi n| 
model poi nt s i n t he f ol 1 ovi ng vay: 

x' = a + a 22/1 +a 3*2 +« 4 
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y' = b 4 x 4 +b 2 yi + b 3 x 2 + b 4 ( 6 ) 

Subs t i t ut i ng f c^rwe obt ai n 

x' = a 12:1 +a 22/1 +a 3 ( s ti 1 x 1 +s r^i/i +s ri 3 2 q +/ +a 4 

y' = b 4 x 4 +6 22/1 +6 3(s ? 1*1 +sr 12 yi +s r 13 z 4 +t x ) +b 4 (7) 

and r e ar r angi ng t he s e e quat i ons we obt ai n 

x' = (a 1 +a 3 sr 11 )x 1 +(a 2 +a 3 sr 1 2 )yi+(a 3 sr 13 )z 1 +(a 3 t x +a 4 ) 

y' = (b 1 +b 3 sr 11 )x 1 +(b 2 +b 3 sr 1 2 )y 4 +(b 3 sr 13 )z 1 +(b 3 t x +a 4 ) (8) 

Usi ng t hes e equat i ons we can der i ve al 1 t he par arret er s of t he t r ans f or mat i on bet ween t he m 
and t he i imge. As s une t he i imge i s obt ai ned by a r ot at i on U, t r ap^larrld rsicdl i ng.s 
Usi ng t he or t honor iml i t y cons t r ai nt we can hr s t der i ve t he s cal e f act or 

s n = ( ®i +® 3 s in) 2 +( ci2 +® 3 s q.2) 2 +( a 3 s ri 3 ) 2 

= a\ + a\ +als 2 +2a 3 s(qr 41 +a 2 r 12 ) (9) 

FromEquati ons (8) and (9), by der i vi ng t he component s of t he t r ans 1 at i pp we cctaar , t 
obt ai n t he pos i t i on of t he robot i n t he i imge r el at i ve to its posi t i on i n t he model vi ews 


Ax = a 3 t x + a 4 


Ay 

Az 


b 3 ty +6 4 



1 

5 


( 10 ) 


Not e t hat ZN: is der i ved f r omt he change i n s cal e of t he obj ect. 
Pi and P' is gi ven by 


u 44 = 

a 1 +a 3 s r 41 

“12 = 

a 2 +a 3 s r 12 

“13 







b\ +a 3 s V 2 \ 


b2 +a 3 S V22 


“21 = 


“22 = 


“23 




The rot ati on mat ri x U betw 


n 3 

S'ri 

b 3 S V2 3 


( 11 ) 


As was al ready ment i oned, t he pos i t i on of t he robot i s comput ed here r el at i ve t o t he posi t i 
t he carter a when t he hr s t model i magq, Pas acqui red. Ac and ZN: r epr es ent t he mot i on of 
t he robot f r 0114 P o P, and t he r es t of t he par amet er s r epr es ent its 3Dr ot at i on and el evat i 
To obt ai n t he r el at i ve posi t i on t he t r ansf or mat i on par amet er s betweenthe mg (held vi e ws , P 
P 2 , are requi red. 


3.3 Repositioning 

An i nt eresti ng vari ant of the positioning problerq referred to as repositioning , is def 
f ol 1 ows . Q ven an i mage, cal 1 ed t he t arget i mage, pos i t i on your self inthe locationfror 
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t hi s i imge ms obs er ye'd One my t o s ol ve t hi s pr obi emi s t o ext r act t he exact posi t i on f r or 
whi chthe target i imge was obt ai ned and di r ect t he robot tot hat position. Inthis secti 
are interestedin a nor e qual i t at i ve approach. Under t hi s approach pos i t i on i s not conpi 
Instead, the robot observes the environment and extracts only the direction to the t; 
1 ocat i on. Uni i ke t he exact approach, t he met hod pres ent ed her e does not r equi re t he r ecc 
of the transf or mat i on between the model views. 

W as s ume we are gi ven wi th a model of t he envi r onment t oget her wi t h a t ar get i mage. 
The robot is al 1 owed tot ake new i mages as it is movi ng t omr ds t he t ar get. W ass ume ; 
hor i zont al 1 y movi ng pi at f or m (I n ot her wor ds , we as s ume t hr ee degrees of f r eedomr at her 
six; t he r obot is al 1 owe d t o r ot at e ar ound t he ve r t i c al axi s and t r ans late hor i z ont al 1 
val i di t y of t hi s cons t r ai nt is di s cus s ed i n Sect i on 6. ) Bel ow we gi ve a s i npl e conput at i ( 
det er ni nes a pat h whi ch term nat es i n t he t ar get 1 ocat i on. At each t i me s t ep t he r obot acqi 
a new i mage and al i gns it wi t h t he model . By conpar i ng t he al i gnment coeffci ent s wi t h t h< 
coefficients for the target image the robot deternines its next step. The algorithmis di 
i nt o t wo s t ages . Inthehrstst age t he r obot hxat es on one i dent i habl e poi nt and moves a] 
a ci r cul ar pat h around t he hxat i on poi nt unt i 1 t he 1 i ne of si ght to t hi s poi nt coi nci d< 
t he 1 i ne of si ght to t he cor r es pondi ng poi nt i n t he t ar get i mage. I n t he second s t age t he 
advances forwardor retreats backmrduntil it reaches the target location. 

Q ven a model compos ed of t wo i magesj aflfd P 2 , I 2 i s obt ai ned f r oqiPy a rot at i on 
about t he Y- axi s by an angl e a , hor i zont al t r ang) cahidonrfal e factor s. Q ven a t ar get 
i mage i?, It is obt ai ned f r ongPy a si ni 1 ar rot at i on by an angl e 9, t r ans^akhdrs ial e 
Sf. Usi ng Eq. (4) t he posi t i on of a t ar get p,oi^)itc (lm be expr es s ed as 


Xf = a 4X4 + a 3^2 +a 4 

Vt = b 2 yi (12) 

( The rest of the coeffcients are zero si nee the platf or mmoves hor i zont al 1 y. ) I n f act, t he 
ci ent s are gi ven by 


a\ 


a 3 


a 4 

b 2 


s t s i n( a — 0 ) 

s i n a 
s t s i n 0 

s si n a 

t x s t s i n 0 

11 ; 

s si no 

s t 


(13) 


( The der i vat i on i s gi ve n i n t he Appendi x. ) 

At everytime stepthe robot acqui res an i mage and al i gns it wi t h t he above model . As s ur 
t hat i mage jPi s obt ai ned as a res ul t of a rot ati on by an angl e <f>, t r^gisalnalt k oml^-s 

J This problem can be cons i der ed as a var i ant of t he honi ng pr obi em A di s cus si on of the gener al horti ng 
pr obi emwi t h a “si gnat ur e- bas ed” s ol ut i on c an be f ound i n[l 1 ]. 
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The posi t i on of a poi nj,, (y) is expr es s ed by 


x p 

y P 


where the coefficients are gi ven by 


Cl 


C3 


c 4 
C?2 


C 1Z1 +C 3S2 +C 4 

d 22/1 


Sp s i n( a — <^>) 

s i n a 
s p s i n cf) 

5 sina 

t x s p sin(f> 

t p 

s si na 

^ p 


(14) 


(15) 


The step performed by the robot is deterninedby 

S _ -1 _ 

C3 d3 


(16) 


That 


l s. 


6 = 


s sin(a- sin(a- 


0) . . 

= s sina( cot cp — 


cot 6) 


(17) 


s i n cj) s i n 6 

The robot shoul d nownove so as to reduce the absol ute val ue of 8 . The di recti on of moti 
depends on t he si gn of a. The robot can deduce t he di recti on by movi ng si i ght 1 y t o t he s 
and checki ng i f t hi s mot i on r es ul t s i n an i ncr eas e or deer eas e of 8 . The mot i on i s defin 
f ol 1 o vs. The r obot moves t o t he r i ght (or to the left, dependi ng on whi ch di recti on reduce 
by a step Ac. 


A new i mage P n i s now acqui red, and t he fixat ed poi nt is locatedinthis i mage. Etenot 
its newposi t i on by. xSi nee t he mot i on i s par al 1 el t o t he i mage pi ane t he dept h val ues of t 
poi nt i n t he t wo vi e vsp, afld P n , ar e i dent i cal . W now want to rot at e t he earner a s o as to 
r et ur n t he fixat ed poi nt t o i t s or i gi nal posi t i on. The angl e of rot at i on, /3, can be deduce 
t he equat i on 

x p =x n cos (3 +si n/3 (18) 

Thi s equat i on has t wo sol ut i ons . W chose t he one t hat count er s t he t r ansi at i on (namel; 
t r ans 1 at i on i s to t he r i ght, t he c ame ra shouldrotate to the left), and t hat keeps t he a 
rotation small. In the next time step the ne w pi„ct apfeaPes pPknd the procedure is 
r epeat ed unt i 1 8 vani s hes . The r es ul t i ng pat his ci r cul ar around t he poi nt of focus . 

Once t he r obot ar ri ves at a posi t i on f or whi ch 8 = 0 (namel y, its 1 i ne of si ght coi n 
vi t h t hat of t he t ar get i mage, and cj) = 9) it should now advance f or var d or r et r eat backva: 
t o adj us t its posi t i on al ong t he 1 i ne of si ght. Sever al measures can be us ed t o det er ni 
di recti on of moti on; one exanpl e i s the^ef raid ch sati sfies 

Cl _ Sp 

ai s t 

when t he t vo 1 i nes of si ght coi nci de. The obj ect i ve at t hi s s t age is t o br i ng t hi s me as ur < 
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4 Handling Iferspective Distortions 


The 1 i near coirbi nati on scheme presented above accuratel y handl es changes i n vi ewpoi nt ass 
i ng t he i mages are obt ai ned under weak per s pect i ve proj ecti on. Error anal ysi s and experi n 
r es ul t s demons t r at e t hat i n many pr act i cal cas es t hi s ass unpt i on i s val id. In cases wher 
s pect i ve di s t or t i ons are toolarge tobe handl ed by a weak per s pect i ve appr oxi mat i on, mat i 
bet ween t he model and t he i mage can be f aci litated in two ways . Che pos si bi 1 i t y i s to ai 
cas es of 1 ar ge per s pect i ve di s t or t i on by augment ingthelibraryof storedmodels with add: 
model s . In a r el at i vel y dens e 1 i br ar y t her e us ual lyexists a model t hat is relatedtoth 
by a s uffci ent 1 y s mal 1 t r ansf or mat i on avoi di ng s uch di s t or t i ons . The second al t er nat i ve 
i nprove t he mat chbetveenthe model and t he i mage usi ng an i t er at i ve pr oces s . I n t hi s sect 
ve consi der the second opti on. 

The s ugges t ed iterativeprocess is basedona Tayl or expans i on of t he per s pect i ve co 
nates. As described below, this expansion results in a pol ynoni al consisting of term 
of whi ch can be appr oxi mat ed by 1 i near combinations of vie vs. The first term of this ser 
represents t he or t hogr aphi c appr oxi mat i on. The process resembles a method of matching 
points vi t h 2D poi nts described recently by EteMnt hon and Ehvi s [ 4] . Inthis case, hove\ 
t he met hod i s appl iedto2Dmodels r at her t han 3D ones . I n our appl i cat i on t he 3Dcoor di nat 
of t he model poi nt s are not pr ovi ded; i ns t ead t hey ar e appr oxi mat ed f r omt he model vi e vs . 

An i mage poi nt ( x , y) =(f X/ Z, fY/Z) is t he pr oj ect i on of s ome obj ect poi nt, (X , Y, 
t he i mage, where / denot es t he focal 1 engt h. Cbnsi der t he f ol 1 ovi ng Tayl or expans i on of 
ar ound s ome de pt h val ueo^ 


- = E 

z ^ 


f {k) ( z>) 


ha 


ki 


( z- 


1 y, ( - 1 k ) ( Z- g) k 


Z Q 

1 

E 


A=i 


(k - 1 )! Z^ +1 


( 20 ) 


1 + f : XA( Z — 

m z » 


The Tayl or series des cr i bi ng t he posi t i on of a poi nt sis t her ef or e gi ven by 


x = 


fX fX 


z 


Z n 


i + E 


- t) (Z- 


fc=l 


(k - i)\j z t 


( 21 ) 


Not i ce t hat t he zero t er mcont ai ns t he or t hogr aphi c appr oxi mat i on f or x. fc$itd±es by A 
fcthtermof the series: 

z 0 (k- i)h z 0 ) 1 ’ 

A r e c ur s i ve de fini tionof the above series is givenbelow. 
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Initialization: 


x (o) = A (o) = IA 

Zq 

Iterative step: 

Z ~ % 

(k - l)oZ 
l^- 1 ) _|_ A ( k ) 

where r epr es ent s t he kt h or der appr oxi imt i on f or arecjaAes ent s t he hi ghes t or der 
t er mi n iP. 


A ( fc ) 
x {k) 


Accor di ng t o t he or t hogr aphi c appr oxi nat i on bot h X and Z can be expr es s ed as 1 i near con 
bi nat i ons of t he model vi ews ( Eq. (4)). W t her ef or e appl y t he above procedure, approxina/ 
X and Z at ever y s t ep usi ng t he 1 i near coirbi nat i on t hat best al i gns t he no del poi nt s wi t h 
i nage poi nt s . The gener al i dea i s t her ef ore t he f ol 1 owi ng. Fi r s t, (^fcaaidti^fiJtfeya: 
s ol vi ng t he or t hogr aphi c case. Then at each s t ep of t he i t er at i on we i nprove t he es t i nat 
s eeki ng t he 1 i near combi nat i on t hat best es t i nates t he f act or 


z - z _ x - 

( k - 1 ) 0 "aPT 


(23) 


Etenot e by xG Tl t he vect or of i nage poi nt coor di nat es , and denot e by 


P = [ xi, jf, 3, 1] 


(24) 


an n X 4 nat r i x cont ai ni ng t he pos i t i on of t he poi nt s i n t he t wo model i nages . Etenot e 
P+ = (pTp)- 1 P T the ps eudo-i nver s e of P (we assume P is over de term ned). Also denote 
by the coefficients computed for the ActhstejP) rfikpresents the linear combination 
computed at that s t ep t o appr oxi mat e t he X or the Z values. Since at ev%^yfs,t ajidZ 
k are const ant t hey can be merged i nt o t he 1 i near combi nat i on. Ete^btaaicbjA^ t he 
vect or s of conput ed val ues of x and Aat the Acthstep. Aniterative procedure t o al i gn a mo 
t o t he i nage is des cri bed bel ow. 


Initialization: 

Solve the orthographic approximation, namely 

a (°) = p + x 

x(°) = A(°) = Pa(°) 

Iteratire step: 

= (x- 

a ( fc ) = P+ q( fc ) 

A^ = (Pa^) ® 

x ( fc ) = x (fc_1) +A (fc) 
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where the vector operations ® and-7 are defined as 


u ® v 

U~r V 


(-ml-ci, . . , n p 4 ^ 

f ^1 ^ 71 \ 

5 • • “ '•>) 

Vl V n 


5 Rejection Mdel — Error Analysis 


Inthis secti on to esti nut e t he er r or obt ai ned by usi ng t he 1 i near combi nat i on net hod. 
met hod ass umes a weak per s pect i ve pr oj ect i on model . W compare t hi s ass umpt ion wi t h t he 
more accurate perspective proj ection model . 

Apoi nt (X , Y, Z) is proj ect ed under t he per s pect i ve model to (r, y) =( f X/ Z, fY/Z)Y 
image, where / denotes the focal length. Under our weak per specti ve model the same poi 
i s appr oxi mat ed by ( a; , y) =(s X, sY ) where s is a s cal i ng f act or . The best esti mat e f or 
s cal i ng f act or , is gi ven by so? jvl/eT e #i s t he aver age dept h of t he observed envi r onment. 
Etenote the error by 

E = \ x — jc (25) 


The error is expres sed by 


Changi ng t o i mage c oor di nat e s 


or 


E= fX(- 

£o 


E= xZ(— 
£o 


E = 1*1 -1T- 
^0 



(26) 

(27) 

(28) 


The er r or is s mal 1 when t he me as ur ed f eat ur e is cl os e t he opt i cal axi s , or when our esti 
for the depth,o,%is close to the real depth, Z. This supports the basic i nt ui t i on t hal 
i mages wi t h 1 owdept h vari ance and f or fixat ed r egi ons (r egi ons near t he cent er of t he i ma 
t he obt ai ned per s pect i ve di s t or t i ons are r el at i vel y s mal 1 , and the systemcantherefore 
t he s cene wi t h hi gh accur acy. Fi gur es 1 and 2 s howt he dept h rbisoaif/hifict i on of x f or 
e = 10 and 20 pi xel s , and Tabl e 5 s hows a number of exampl es for t hi s f unct i on. The al 1 owe 
depth vari ance, ZJ,Z\ s computed as a functi on of x and thetoleratederror, e . For exampl 
a 10 pi xel error toleratedinafieldof viewof up to ±50 pi xe Is is equi val ent to al 1 owi nj 
vari at i ons of 20%. Fr omt hi s di s cus si on i t is apparent t hat when a model is al i gned t o t he 
t he r esul t s of t hi s al i gnment shoul d be j udged di ffer ent 1 y at di ffer ent poi nt s of t he i magt 
f art her away a poi nt i s f r omt he cent er t he more di s cr epancy s houl d be tolerated between 
predi cti on and the actual i mage. Afive pi xel error at posi ti on x =50 i s equi val ent to a 10 
error at positions; =100. 


So far we have consi dered the di screpanci es betweentheweakperspective and the persp 
t i ve proj ect i ons of poi nt s . The accur acy of t he TC s cheme depends on t he val i di t y of t he ' 
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Fi gure 1:J^- as a functi on of x for e =f0 pi xel s 



Fi gure 2:-S- as a functi on of x for e =20 pi xel s 






x \ e 

5 


10 


15 


20 


25 

1 . 

2 

1 . 

4 

1 . 

6 

1 . 

”8 

50 

1 . 

1 

1 . 

2 

1 . 

3 

1 . 

4 

75 

1 . 

07 

1 . 

13 

1 . 

, 2 

1 . 

, 27 

100 

1 . 

05 

1 . 

1 

1 . 

15 

1 . 

, 2 


Tabl el: AL1 owed dept h r at ijfg, as a f unct i on of x (hal f t he wi dt h of t he hel d consi der ed 
and the error al 1 owed (e , i n pi xel s ). 


perspective projection bot h i n t he no del vi ews and f or t he i nconi ng i nage. Intherestof 
s ect i on we devel op an er r or t er mf or t he LC s chene ass uni ng t hat bot h t he model vi e ws anc 
t he i nconi ng i nage are obt ai ned by per specti ve proj ecti on. 

The error obt ai ne d by us i ng t he LC s c he me i s gi ve n by 

E = \x — a$— by— c§— $ (29) 


Si nee t he s chene accur at el y pr edi ct s t he appear ances of poi nt s under weakperspective pr 
t i on, it s at i s hes 

x =ax \ — b{y— eje— d (30) 

whe re accented letters represent orthogr aphi c appr oxi nat ions. Ass ume that inthe two no 
pi ct ur e s t he dept h r at i os ar e r oughl y e qual : 


yM y 7 

_ ^01 ^ ^02 

~zfi ~ zT ~ ~Zt. 


(31) 


( Thi s condi t i on i s s at i shed, for exanpl e, when betweenthetwo model i mages t he earner a or 
t r ans 1 at es al ong t he i nage pi ane. ) Us i ng t he f act t hat 


_fX_ fXZ 0 Z 0 
Z Z 0 z x Z 


(32) 


we obt ai n 


E 


\x — a$— by— c§ — 


Z n 


yM 

',_ 0 _ 

Z M 


/M 

Z 0 


x —— a p 
Z 

x ~/~ ( a i x ~ biy- 

7 yM 

Xy^- - ( X - - I 

Zj 


> Z M 

yM 

c &) Z M 


yM 

^0 7 

C $/M- d 

- d 


(33) 
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7 r/M 

Zjq Zj q 


■M 

0 

■M 


1 ) 


< 


y yM 

~z ~ ~z™ 


\d\ 


yM 

Z 0 

z M 


1 


The er r or t her ef or e depends on t wo t er he . The first gets s iml 1 er as t he i imge poi nt s get cl 
t o t he cent er of t he f r ame and as t he di ffer ence bet ween t he dept h r at i os of t he model and ' 
i mage gets s mal 1 er . The second gets s mal 1 er as t he t r ansi at i on component gets s mal 1 er ar 
the model gets close to orthographi c. 

Fol 1 owi ng t hi s anal ysi s , weak per spet i ve can be used as a pr oj ect i on model when t he de 
var i at i ons i n t he s cene are r el at i vel y 1 ow and when t he s ys t emconcent r at es on t he cent er 
of t he i mage. W concl ude t hat, by fixat i ng on di s ti ngui shed part s of t he envi ronment, 

1 i near combi nati ons scheme can be used for 1 ocal i zati on and posi ti oni ng. 


6 Imposing Constrairts 

Local i z at i on and posi t i oni ng r equi re a 1 ar ge memor y and a great deal of on-1 i ne conput al 
A large number of models mist be stored to enable the robot to navigate and manipulate 
in relatively large and complicated environments. The computational cost of model - i i 
conpar i s on i s hi gh, and if cont ext ( s uch as pat h hi s t or y) is not avai 1 abl e t he number of rei 
compar i s ons may get ver y 1 ar ge. To reduce t hi s conput at i onal cost a number of cons t r ai nt s r 
be employed. These constraints take advantage of the structure of the robot, the propert 
i ndoor envi r onment s , and t he nat ur al pr oper t i es of t he navi gat i on t as k. Thi s secti on exa 
some of these constraints. 

One t hi ng a s ys temmay at tempt t o do i s to bui 1 d t he set of model s s o as to reduce t fi 
effect of perspective distorti ons i n order to avoi d performing iterative conput ati ons. 
of the environment obtained when the systemlooks relatively deep into the scene usu; 
s at i s f y t hi s condi t i on. \Hen per s pect i ve di s t or t i ons are 1 ar ge t he s ys temmay consi der mo 
s ubs et s of views relatedbyatr ansi at i on par al 1 el t o t he i mage pi ane (per pendi cul ar t o 1 
of sight). In this case the depth values of the points are roughly equal across all 
consi dered, and i t can be shown that novel vi ews can be expressed by 1 i near combi nati ont 
t wo model vi ews even i n t he pr es ence of 1 ar ge per s pect i ve di s t or t i ons . Thi s becomes appi 
f r omt he f ol 1 owi ng der i vat ion. bpt^(^),l< i < n be a poi nt pr oj ect ed i n t he i mage 
to ($, y) = fY/Y ), and 1 et {(,xf) be the pr oj ect ed poi nt after applying a ri gi d 

t r ans f or mat ion. Ass uni ng t h^t= Z ; we obt ai n 

ZiX- = r uXi -|-r 12 Y Yx 13 Yi Yt x 

Ziy'i = x 21 Xi Yx 22 Y Yx 23 Zi Yt y ( 34 ) 

n vi di ng by ^Zwe obt ai n 

, 1 

x t = r 11 x t Yx 12 yi Yx 13 Yt x — 
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1 

r 2 \x t +r 22 m +r 23 +t y — 


(35) 


Vi = 


Rewritingthis invector equat i on f or mgi ves 

x' = r n x+r i 2 y+r i 3 l+t ^z -1 

y' = r 2 ix+r 22 y + r 23 1-K ^z -1 (36) 

where x, y, i, and f are the vector s c$ x/, and $ val ues respecti vel y, 1 i s a vector 
of all Is, and 1 zis a vector of f/vSlues. Consequently, as i n t he weak per s pect i ve cas e, 
novel vi evs obt ai ned by a t r ansi at i on par al 1 el t o t he i nage pi ane can be expr es sed by 1 i 
combi nations of four vectors. 

An i ndoor environment us ual 1 y pr ovi des the robot with a flat, horizontal support. Cbn 
quent 1 y, t he mot i on of t he earner a is of t en cons t r ai ned t o rot at i on about t he vert i cal (Y 
and to transl ati on i n the XZ- pi ane. Such moti on has onl y three degrees of f reedomi nstea 
t he si x degrees of f reedomi n t he gener al case. Under t hi s cons t r ai nt fever cor r es pondenc 
r equi r ed t o al i gn t he model vi t h t he i mage. For exampl e, i n Eq. (4) ( above) t he coeffici e 
a 2 =b i =b 3 =b 4 = 0. Three poi nt s r at her t han f our are r equi redtodeternine the coefficient 
by s ol vi ng a 1 i ne ar system Two, ratherthanthree, are re qui r e d i f t he quadr at i c c ons t r ai 
al s o cons i der ed. Anot her advant age t o consi deri ng onl y hori zont al mot i on i s t he f act t he 
mot i on cons t r ai ns t he pos si bl e epi pol ar lines betveeni mages . Thi s fact can be usedto 
the task of correspondence seeki ng. 

Obj ect s ini ndoor envi r onment s s omet i mes appear i n r oughl y pi anar settings. In parti c 
the relatively static objects tend to be located along vails. Such objects include vi 
shelves, pictures, closets and t abl es . lien t he ass umpt i on of or t hogr aphi c pr oj ect i on i 
(for exampl e, when t he r obot is r el at i vel y di s t ant f r omt he val 1 , or when t he 1 i ne of si 
roughl y perpendi cul ar to the val 1 ) the transf ormati on betveen any t vo vi ews can be descri 
by a 2D affiie t r ansf or mat i on. The di mensi on of t he s pace of vi ews of t he s cene i s t hen re due 
tothree (r at her t han f our ), and Eq. (4) becomes 

x' = a 1 X 1 + a 2 yi +a 4 1 

y' = b 1 X 1 +b 2 yi +b 4 1 (37) 

( =b 3 = 0 . ) Onl y one viewis therefore s uffici ent t o model t he scene. 

M>s t office -1 i ke i ndoor envi r onment s ar e compos ed of rooms connect ed by cor r i dor s . Nav 
gat i ng i n such an envi r onment i nvol ves maneuveri ng t hr ough t he cor ri dor s , ent eri ng and exi 
t he rooms. Not al 1 poi nt s i n s uch an envi ronment are equal 1 y i report ant. Juncti ons , pi aces e 
t he robot f aces a nunber of opti ons f or changi ng i t s di recti on, are more i mport ant t han c 
places for navigation. In an indoor environment these places include the thresholds of 
and t he begi nni ngs and ends of corridors. Anavi gati on systemvoul d therefore tend to s 
more model s for t hes e poi nt s t han f or ot hers . 

One i mpor t ant pr oper t y s har ed by many j unct i ons i s t hat t hey are confined to r el at i ve 
s mal 1 areas . Consi der f or exampl e t he t hr es hoi d of a room It is a r el at i vel y nar r ow 
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that separates the roomfromthe adjacent corridor. Wen a robot is about to enter a roc 
a comron behavi or i ncl udes s t eppi ng t hr ough t he door , 1 ooki ng i nt o t he r oorg and i dent i f 3 
it before adecisionis made to enter the roomor to avoi d i t. The set of i nt er es t i ng i ma£ 
t hi s t as k i ncl udes t he set of vi ews of t he r oomf r omi t s ent r ance. Pr ovi ded t hat t hr es hoi 
narrowt hese vi ews are r el at ed t o each ot her al most excl usi vel y by rot ati on around t he ve: 
axi s . finder per s pect i ve pr oj ect i on, s uch a r ot at i on i s r el at i vel y eas y t o recover . The 
of points i n novel views can be recovered fromone model viewonly. This is apparent fr 
t he f ol 1 ovi ng deri vat i on. Consi der a poi nt p = ( X, Y, Z). Its posi t i on i n a model vi ewi t 
by ( x , y ) =( f X/ Z, fY/Z). Now, cons i der anot her vi ew obt ai ned by a r ot at i on R around t 
carter a. The 1 ocati on of p i n the newvi ewi s gi ven by (assurri ng / =1) 


rnX+r 1 2 Y +r 13 Z r 21 X+r 22 Y +r 23 Z 
r 31 X+r 32 Y +r 33 Z’r 31 X+r 32 Y +r 33 Z 


(38) 


i npl yi ng t hat 

, / ,s J 11 X +r 12 y +r 13 r 21 x +r 22 y +r 23 

(x, y) =(-,-) (39) 

rsix +r 32 y +r 33 r 31 x +r 32 y +r 33 

Etept h i s t her ef or e not a f act or i n det er ni ni ng t he r el at i on bet ween t he vi ews . Eq. ( 39) be' 
even si npl er i f onl y r ot at i ons about the T- axi s ar e consi der ed: 


(^, i) 


x cos «+sina y 

— rsina+cosV xsina+cosa 


(40) 


where a i s t he angl e of rot at i on. I n t hi s cas e ct can be recovered merely froma si ngl e c 
s pondence. 


7 Bcperinarts 


The TC met hod was i npl ement ed and appl i ed t o i mages t aken i n an i ndoor envi r onment. 

I mages of t wo offces , Aand B, t hat have sinilar structures weret aken usi ng a Panas oni c camei 
withafocal 1 engt h of 700 pi xe Is. Seni-staticobjects, suchas heavy f ur ni t ur e and pi ct ur 
us ed t o di s t i ngui sh between the offces . Fi gur e 3 s hows t wo model vi e ws of offce A The vi e ws 
were taken at a distance of about 4mf r omt he wal 1 . Cbr respondences were pi eked manual 1 
The r es ul t s of al i gni ng t he model vi ews t o i mages of t he t wo offces ar e pres ent ed i n Fi gur 
The left i mage cont ai ns an over 1 ay of a pr edi ct ed i mage (t he t hi ck whi t e 1 i nes ), cons t r uc 
1 i near 1 y combi ni ng t he t wo vi ews , and an act ual i mage of offce A A good mat ch between the 
two was achi eved. The ri ght i mage cont ai ns an overl ay of a predi cted i mage constructed f ] 
a model of offce B and an i mage of offce A Because the offces share a si ni 1 ar structure the 
static cues (the wall corners) were perfectly aligned. The seni-static cues, however, 
mat ch any f eat ur es i n t he i mage. 

Fi gur e 5 s hows t he mat chi ng of t he model of offce A wi t h an i mage of t he s ame offce ob- 
t ai ne d by a r el at i vel y 1 ar ge mot i on f or war d ( about 2m) and t o t he s i de (about 1. 5m). A t ho 
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Figure 3: Two model views of offce A 



Fi gur e 4: Mt clii ng a no del of offce At o an i lmge of offce A (1 ef t ), and mat chi ng a no del of 
offce B t o t he s a me i lmge ( r i gilt ). 



Fi gure 5: Mf chi ng a nodel of offce Ato an i lmge of the sane offce obt ai lied by a rel ati vel y 
1 ar ge not i on f or war d and to t he r i gilt. 
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Figure 6: Two nodel views of a corridor. 



Figure 7: Matching the corridor nodel with two inages of the corridor. Hie right inage v 
obtained by a relatively large notion forward (about half of the corridor length) and t 
r i gilt. 


the distances are rel ati vel y short nost perspective distortions are negligible, and a go 
between the nodel and the i nage i s obt ai lied. 

Another set of inages was taken in a corridor. Here, because of the deep structure 
the corridor, perspective distortions are noticeable. Nevertheless, the alignment re: 
demonstrate an accurate nat cli i n 1 arge porti oils of the i nage. Fi gure 6 shows two nodel vi e 
of the corridor. Figure 7 (left) shows an over 1 ay of a linear conbi nat i on of the nodel 
wit hail image of the corridor. It can be seen that the parts that ar § r el at i vel y di s t am 
perfectl y. Fi gure 7 ( ri gilt ) shows the nat chi ng of the corri dor nodel wi tli an i nage obt ai lie 
a rel ati vel y 1 arge not i on ( about half of the corridor length). Because of perspective di s 
t he rel ati vel y near feat ur es no 1 onger al i gn ( e. g. , t he near door edges ). The rel at i vel y 
liowe ve r , still nat c h. 

Hie next experiment shows the application of the iterative process present ed i n Sect: 
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in cas es where 1 ar ge per s pect i ve di s t or t i on were not i ceabl e. Fi gur e 8 s hows t wo no del v 
and Figure 9 shows the results of Hatching a linear combination of the model views to 
i nage of t he s ame offce. I n t hi s cas e, becaus e t he i nage was t aken froma relatively cl 
di s t ance, per s pect i ve di s t or t i ons cannot be negl ect ed. The effects of perspectivedistom 
be not i ced on t he r i ght corner of t he boar d, and on t he edges of t he hanger on t he top r i j 
Perspective effects were reduced by using the iterative process. The results of appl yi r 
pr ocedur e af t er one and t hr ee i t er at i ons ar e s hown i n Fi gur e 10. 

The experi merit al r es ul t s demons t r at e t hat t he TCmet hod achi eves accur at e 1 ocal i zat i o 
many cas es , and t hat when t he met hod f ai 1 s becaus e of 1 ar ge per s pect i ve di s t or t i ons an i t e 
conput at i on can be us ed t o i nprove t he qual i t y of t he mat ch. 


8 G)ixl im oik 

Amethod of 1 ocal i zati on and posi ti oni ng i n an i ndoor envi ronment was presented. The meth 
i s based on representi ng the scene as a set of 2Dvi ews and predi cti ng the appearance of r 
views by linear combinations of the model views. The method accuratel y approxi mates t. 
appearances of scenes under weak per s pect i ve projection. Analysis of this projection, 
as experimental results demonstrate that in many cases this approximation is suffcienl 
accur at el y des cr i be t he s cene. \Hen t he weak per s pect i ve approxi mat i on i s i nval id, ei t 
1 ar ger number of model s can be acqui red or an i t er at i ve s ol ut i on can be empl oyed to acco 
for the perspective distortions. 

The met hod presentedin this paper has s ever al advant ages over exi s t i ng met hods . It u 
rel ati vel y ri ch representati ons ; t he r epr es ent at i ons are 2D r at her t han 3D, and 1 ocal i z a 
be done f r oma s i ngl e 2D vi ew onl y. The s ame basi c met hod i s us ed i n bot h t he 1 ocal i z at i 
and posi t i oni ng pr obi errs , and a s i mpl e al gor i t hmf or repositi oni ng i s der i ved f r omt hi s m 
Future work i ncl udes handl i ng the probl emof acqui si ti on and mai ntenance of model s , devel 
i ng effei ent and r obus t al gor i t him for s ol vi ng t he cor r es pondence probl erq and bui 1 di ng r 
us i ng vi s ual i nput. 


^perrfix 


I n t hi s appendi x we der i ve t he expl i ci t val ues of t he coeffci ent s of t he 1 i near combi nat i ons 
cas e of hor i zont al mot i on. Consi der a poi nt p =(x, y, z) t hat is pr oj ect ed by weak per s p 
to t hr ee i mages i,Pi 2 , and P, pis obt ai ned f r ongI?y a rot at i on about the F- axi s by an 
angl e a, t r ansi ati^p^i dnd s cal e f act sand P i s obt ai ned f r ongl? rot ati on about t he 
Y - axi s by an angl e 0 , t r ansi qtanxi rfcal e^s The posi t i on of p i n t he t hr ee i mages is gi ven 
by 


Oi, u) = (x, y) 
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Figure 10: The results of applying the iterative process to reduce perspective distorti 
one (left) and three (right) iterations. 
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(x 2 , u) = ( S m x cos a +s m z si n a +p, 

( J, {/')=( s p x cos 9 si n 9 +jf, ) 

The poi nt ('% jj) can be expr es s ed by a 1 i near coirbi nat i on of t he fir s t t wo poi nt s : 

x' = a\X\ + a 2*2 + a 3 
/ = b yi 

Rewri ti ng these equati ons we get 

s p x cos 9 -\~fz si n 9 = «ir +ffi2(4n ;c cos a +4j/; si na + 4 ) + a 3 

s P V = by 

Equat i ng t he val ues for t he coeffci ent s i n bot h s i des of t hes e equat i ons we obt ai n 


SpCos 9 = +a 2 S m cos a 

s p s i n 9 = qs m s i n a 

tp = a 2tm ~\~ a 3 


and t he coeffci ent s are t heref ore gi ven by 


ai 


a 3 


a 4 
b 


Sp s i n( a — 9 ) 

s i n a 
s p si n 9 

s m s i n a 

t m s p sin9 

t p 

s m s i n a 
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